An organic brain-inspired platform with neurotransmitter closed-loop control, actuation and reinforcement learning

Organic neuromorphic platforms have recently received growing interest for the implementation and integration of artificial and hybrid neuronal networks. Here, achieving closed-loop and learning/training processes as in the human brain is still a major challenge especially exploiting time-dependent biosignalling such as neurotransmitter release. Here, we present an integrated organic platform capable of cooperating with standard silicon technologies, to achieve brain-inspired computing via adaptive synaptic potentiation and depression, in a closed-loop fashion. The microfabricated platform could be interfaced and control a robotic hand which ultimately was able to learn the grasping of differently sized objects, autonomously.


Supplementary Figure S1: Sensitivity range of the ENODe upon repeated oxidation of dopamine.
Fig. S1, For the open circuit characterization, different concentrations of dopamine were tested and three measurements for each concentration were repeated for three different devices (N=3, 9 measurements in total).The statistics of the channel conductance variations resulting from the first dopamine measurements for each device and the statistics resulting from all performed measurements were reported.For low concentrations of dopamine solutions ([5,10,15,30] µM), the effect due to the first measurement was comparable with the following measurements performed with the same device in terms of values and error bars (differences < 2%).For high dopamine concentrations ([50,75,100] µM), the PEDOT:PSS channel was saturated, as the mean value of conductance variation decreased after the initial oxidation of neurotransmitter (differences > 2%).
Electronic Supplementary Material (ESI) for Materials Horizons.This journal is © The Royal Society of Chemistry 2024 Fig. S4, Schematics of connections of the electrochemical feedback-loop and of the robotic hand.The channel current value of the ENODe was recorded and a response signal was sent to the microfluidic system (left), controlled by the PID control software.After the application of each voltage pulse, the channel current value was sent to Arduino, which controlled five servomotors, assembled in the robotic hand (right).To this end, Arduino was connected to customized PID software (Arkeo Cicci Research srl) and the different motors were connected to the five analogic outputs of Arduino through a board.Each motor was connected through nylon thread to a single finger.When an electrical signal was applied, the angle of the motors varied, resulting in the wire pulling of the fingers.The servomotors were powered by a 7V supplier.

Supplementary Discussion S2: Operation of the feedback-loop control system.
The variable controlled by the feedback-loop control system was the channel current amplitude of the ENODe after the application of a square voltage pulse at the gate terminal.The workflow of the customized software based on lab view is the following: 1) Prior to the initiation of the experiment, these parameters were set as follows: -the value of the drain constant voltage V DS was -0.2V; -the shape of the pulse to be applied as voltage gate was defined with 2s of ON time duration, 6s of OFF time and 0.3V amplitude; -the microfluidic pump that should work depending on the status of the controller was chosen by selecting the sign of the error.For positive errors, the pump with the dopamine solution was activated whereas for negative errors the other pump would be activated.
-the maximum flow rate that the pumps can reach was set to 1 mL/min; -the first desired value for channel current I SET was chosen in the tolerance regime of the device (known from the electrical characterization of the ENODe) and the range of error was set in the order of µA. 2) As the experiment started the microfluidic channel of the ENODe contained PBS solution only.The first pulse was applied to the gate and the channel current I DS was measured after the removal of the pulse.3) The error was calculated as: e(t) = I SET -I DS , where both I DS and I SET had negative values.
-if I SET > I DS , a positive error resulted and the pump for the dopamine solution was initiated with a flow rate that depended on the amplitude of the error and the set values for the proportional, integrative and derivative constants of the PID (as described for the control law in Supplementary Information S5); -if I SET < I DS , a negative error was measured and the pump for the H 2 O 2 solution was activated with a flow rate depending on the error value and the set values for the proportional, integrative and derivative constants of the PID (as described for the control law in Supplementary Information S5). 4) Depending on the OFF time selected of the gate voltage, a new pulse was applied to the ENODe, inducing a variation of I DS associated with the oxidation of dopamine or the action of H 2 O 2 .The workflow was then repeated starting from point (2), considering that I SET could be modified any time during the experiment.

Supplementary Figure S5: Experiment of reinforcement learning and electrical-closed loop on the robotic hand.
Fig. S5, Picture of the robotic hand and the sensor pressure placed on the external phalanx of the middle finger, to detect the gripping of the ball.In this work, the thin pressure sensor was combined with a 10kΩ resistor, to obtain a variable voltage that could be read by a microcontroller's analog-to-digital converter when a force was applied.
To evaluate the reinforcement learning capability of the ENODe coupled to the robotic hand, the customized software based on LabVIEW followed this workflow: 0) Prior to the initiation of the experiment, these parameters were set as follows -the value for the drain constant voltage V DS was -0.2V; -the shape of the pulse to be applied as voltage gate was defined with 2s of ON time duration, 6s of OFF time and 0.3V of amplitude; -the dopamine solution pump was activated.
-the maximum flow rate that the pumps could reach was set to 0.2 mL/min, lower than in the experiments of characterization for the control to avoid unnecessary dopamine solution usage during long measurements (~ 6-8 minutes); -in this experiment the PID controller allowed a continuous flow of dopamine until the electrical control from the pressure sensor was reached or the PEDOT:PSS channel of the ENODe is not completely de-doped, then the desired value for channel current I SET is -10 µA (~0 µA) and the range of error is set in the order of µA.
-a delay time of 10s was set to allow Arduino to process the measured data without interferences with the measurement system.-In this experiment I SET > I DS and a positive error was obtained: the pump for the dopamine solution was activated with a flow rate that depended on the amplitude of the error and the set values for the proportional, integrative and derivative constants of the PID (as described for the control law in Supplementary Information S5). 3) When the value of I DS reached the th_start, the motors of robotic hand rotate.The sensor pressure measurement was then monitored: -if its value was 0, the cycle was repeated and the hand opened again (learning process); -If its value was higher than 0 (some force was applied), the motors would stop in the position they were until the force was removed and the learning process (for the position) was finished.
Fig.S5, Picture of the robotic hand and the sensor pressure placed on the external phalanx of the middle finger, to detect the gripping of the ball.In this work, the thin pressure sensor was combined with a 10kΩ resistor, to obtain a variable voltage that could be read by a microcontroller's analog-to-digital converter when a force was applied.To evaluate the reinforcement learning capability of the ENODe coupled to the robotic hand, the customized software based on LabVIEW followed this workflow: 0) Prior to the initiation of the experiment, these parameters were set as follows -the value for the drain constant voltage V DS was -0.2V; -the shape of the pulse to be applied as voltage gate was defined with 2s of ON time duration, 6s of OFF time and 0.3V of amplitude; -the dopamine solution pump was activated.-themaximum flow rate that the pumps could reach was set to 0.2 mL/min, lower than in the experiments of characterization for the control to avoid unnecessary dopamine solution usage during long measurements (~ 6-8 minutes); -in this experiment the PID controller allowed a continuous flow of dopamine until the electrical control from the pressure sensor was reached or the PEDOT:PSS channel of the ENODe is not completely de-doped, then the desired value for channel current I SET is -10 µA (~0 µA) and the range of error is set in the order of µA.-a delay time of 10s was set to allow Arduino to process the measured data without interferences with the measurement system.-athreshold th_start = 10 μA was set in the Arduino script to indicate for which modulation of the channel current I DS the robotic hand should start the closure movement.-athreshold th_end = 40 μA was defined in the Arduino script to indicate for which values of current modulation the robotic hand should be completely closed.In this way, the different motor angles are mapped on a range of channel currents of the ENODe. 1) As the experiment started, the microfluidic channel of the ENODe contains PBS solution only.After the delay time, the first pulse was applied to gate and the channel current I DS after the removal of the pulse was measured.2) The error was calculated as: e(t) = I SET -I DS , where both I DS and I SET reached negative values.-In this experiment I SET > I DS and a positive error was obtained: the pump for the dopamine solution was activated with a flow rate that depended on the amplitude of the error and the set values for the proportional, integrative and derivative constants of the PID (as described for the control law in Supplementary Information S5). 3) When the value of I DS reached the th_start, the motors of robotic hand rotate.The sensor pressure measurement was then monitored: -if its value was 0, the cycle was repeated and the hand opened again (learning process); -If its value was higher than 0 (some force was applied), the motors would stop in the position they were until the force was removed and the learning process (for the position) was finished.

Fig. S6 ,
Fig. S6, Channel current of ENODes when different values of V DS [-0.6 V ,0.1 V] were applied, for different values of V GS [-0.2 V, 0.8 V], showing a switching off the device for highest values of gate voltage.